仅使用前视投影 得到的效果更好？可能是网络的规模不够大
尝试仅使用前视投影+sn
删除前视投影 仅保留pc分支：
loss非常不稳定 生成图像效果差
将pc分支的浅层特征经过down和res分支
另外将pc特征加入到discriminator中 或者从discriminator中取消投影输入
SN: spectral norm
在discriminator中删除conditional输入之后， feature matching loss和vgg loss的初始值变得非常高，这可能是因为没有conditional输入之后，discriminator在开始接受到的输入就完全是混乱的初始生成图像 没有与真实图像的feature比较类似的条件输入；但是loss下降的趋势还比较稳定(即没有造成训练过程的不稳定)，后期也下降到和有conditional输入的discriminator一样的情况了
最终的测试结果没有加入conditional输入好
目前来看Generator输入的上采样图像的作用比较大
静止场景：
0016 0017 0019（步行街场景 行人多 光照复杂）
尝试noVGG 效果很差
尝试pixel shuffle 解决棋盘格效应
object sensitive loss
尝试对gamma进行随机增强，生成的图像亮度更高，在50轮的时候视觉效果更好一些，但是没有最开始的点云分支好
尝试加入pointnet2的head， 收敛速度变慢， 训练集有一些图像效果还可以，对训练集拟合效果不错
激光雷达+图像 去阴影
和相机重建出来的对比
将测试集换为tracking与object的差集 效果不错
尝试在resblock前加入pc特征 效果比resblock前加入的细节和边缘要好
对点云的intensity进行归一化
在Discriminator中取消SN后面的BN之后 Gan的Feature matching loss的量级下降了很大 从[7,12]下降到了[1, 2]， 这可能是因为SN实现了Lipschitz连续条件 但是只使用SN时拟合效果非常差
无法复现之前pix2pix的效果了 目前的改动：Gamma：无效 flip：无效 batchsize 有一些效果，现在感觉可能是数据集的问题
尝试直接用前视投影的点云提特征？
加入膨胀卷积之后效果很好
远处的弱化 从loss上考虑
LiCAM segmentation 实验
Accumulating evaluation results... │tmpfs 38G 3.8G 34G 10% /run
DONE (t=0.06s). │/dev/sda2 439G 393G 24G 95% /
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.590 │tmpfs 189G 736M 188G 1% /dev/shm
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.882 │tmpfs 5.0M 0 5.0M 0% /run/lock
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.646 │tmpfs 189G 0 189G 0% /sys/fs/cgroup
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.402 │/dev/sda1 511M 3.7M 508M 1% /boot/efi
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.689 │tmpfs 38G 28K 38G 1% /run/user/108
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.866 │tmpfs 38G 22G 16G 58% /run/user/1000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.180 │tmpfs 38G 0 38G 0% /run/user/1001
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.613 │(base) ➜ sTrain-sTest_LiCAM_cas101_MS sudo kill 3771
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.625 │[sudo] password for bdbc201:
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.453 │(base) ➜ sTrain-sTest_LiCAM_cas101_MS
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.715 │(base) ➜ sTrain-sTest_LiCAM_cas101_MS du -sh
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.907
LiCAM 测真实图像
DONE (t=0.20s). │/dev/sda2 439G 393G 24G 95% /
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.613 │tmpfs 189G 736M 188G 1% /dev/shm
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.885 │tmpfs 5.0M 0 5.0M 0% /run/lock
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.689 │tmpfs 189G 0 189G 0% /sys/fs/cgroup
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.397 │/dev/sda1 511M 3.7M 508M 1% /boot/efi
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.737 │tmpfs 38G 28K 38G 1% /run/user/108
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.886 │tmpfs 38G 22G 16G 58% /run/user/1000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.195 │tmpfs 38G 0 38G 0% /run/user/1001
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.646 │(base) ➜ sTrain-sTest_LiCAM_cas101_MS sudo kill 3771
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.662 │[sudo] password for bdbc201:
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.458 │(base) ➜ sTrain-sTest_LiCAM_cas101_MS
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.778 │(base) ➜ sTrain-sTest_LiCAM_cas101_MS du -sh
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.932
真实图像测SPADE
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.449
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.678
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.485
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.224
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.582
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.783
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.161
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.481
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.481
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.254
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.610
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.816
真实图像测皮鞋pix2pixHD
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.146
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.301
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.130
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.038
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.179
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.407
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.092
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.168
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.168
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.043
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.209
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.442
真实图像测LiCAM
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.388
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.636
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.407
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.151
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.511
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.809
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.152
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.429
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.431
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.186
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.553
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.847
真实图像测SPADE-LICAM-Intensity
直接High res
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.371
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.586
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.410
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.111
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.502
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.813
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.141
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.402
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.402
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.137
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.532
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.858
Low res 2 High res
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.365
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.583
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.399
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.112
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.496
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.791
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.142
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.398
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.398
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.138
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.530
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.829
目前看应该是需要平滑的conditional input（这个地方可以abalation study）
真实图像测SAPDE-LICAM
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.337
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.550
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.362
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.102
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.453
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.767
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.135
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.376
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.376
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.128
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.494
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.814
真实图像测pix2pix
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.007
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.010
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.010
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.004
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.015
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.003
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.003
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.003
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.003
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.014
真实图像测cycle
None 没有结果
改用OMP多核实现之后 生成1M个点可以从90s提高到10s
使用光栅化过程的zbuffer解决了深度遮挡的问题 可以考虑不用光线追踪 但是仍然是串行实现的
ray tracng 之后的结果会在属于同一个类别的大片联通区域产生孔洞，孔洞中的数值是与该区域的类别非常相似的数值。这在图像中无所谓，
闭运算前
